Surgical positioning system

ABSTRACT

A system for positioning and repositioning of a portion of a patient&#39;s body with respect to a treatment or imaging machine includes multiple cameras to view the body and the machine. Index markers, either light-emitting, passive, geometric shapes, or natural landmarks, are identified and located by the cameras in 3D space. In one embodiment, such reference or index markers are in a determinable relationship to analogous markers used during previous image scanning of the patient. Anatomical targets determined from image scanning can be located relative to reference positions associated with the treatment or diagnostic machine. Several forms of camera, index markers, methods and systems accommodate different clinical uses. X-ray imaging of the patient further refines anatomical target positioning relative to the treatment or diagnostic imaging reference point. Movements of the patient based on comparative analysis of imaging determined anatomical targets relative to reference points on treatment or diagnostic apparatus are controlled by the system and process of the invention.

[0001] This is a continuation-in-part of U.S. patent applications:

[0002] Ser. No. 08/482,213, filed Jun. 7, 1995 by Eric R. Cosman for “AnOptically Coupled Frameless Stereotactic Space Probe,” which is acontinuation of Ser. No. 08/299,987, filed Sep. 1, 1994 by Eric R.Cosman for “An Optically Coupled Frameless Stereotactic Space Probe,”now abandoned, which is a continuation of Ser. No. 08/047,879, filedApr. 15, 1993 for “An Optically Coupled Frameless Stereotactic SpaceProbe,” now abandoned, which is a continuation of Ser. No. 07/941,863,filed Sep. 8, 1992 by Eric R. Cosman for “An Optically Coupled FramelessStereotactic Space Pro,” now abandoned, which is a continuation of Ser.No. 07/647,463, filed Jan. 28, 1991 by Eric R. Cosman for “An OpticallyCoupled Frameless Stereotactic Space Probe,” now abandoned.

[0003] Ser. No. 08/710,587, filed Sep. 16, 1996 by Eric R. Cosman for “AStereotactic Target Localization and Alignment System for the Body,”which is a continuation of Ser. No. 08/275,041, filed Jul. 13, 1994 byEric R. Cosman for “A Stereotactic Target Localization and AlignmentSystem for the Body,” now abandoned.

[0004] Ser. No. 08/795,241, filed Feb. 19, 1997 by Eric R. Cosman for “AHead and Neck Localizer System,” which is a Continuation of Ser. No.08/382,226, filed Jan. 31, 1995, by Eric R. Cosman for “A Head and NeckLocalizer System,” now abandoned.

[0005] Ser. No. 08/779,047, filed Jan. 6, 1997 by Eric R. Cosman for“X-ray Image Machine Assistance in Stereotactic Radiotherapy,” which isa continuation of Ser. No. 08/439,211, filed May 11, 1995 by Eric R.Cosman for “X-ray Image Machine Assistance in StereotacticRadiotherapy,” now abandoned, which is a continuation-in-part of Ser.No. 08/710,587, filed Sep. 19, 1996 by Eric R. Cosman, which is acontinuation of Ser. No. 08/275,041, filed Jul. 13, 1994 by Eric R.Cosman for “A Stereotactic Target Localization and Alignment System forthe Body”. Ser. No. 08/736,495, filed Oct. 24, 1996, by Eric R. Cosmanfor “Repositioner for Head, Neck and Body.”

BACKGROUND AND SUMMARY OF THE INVENTION

[0006] Frameless stereotaxy is widely used in the field of neurosurgery.It involves the quantitative determination of anatomical positions basedon scan data taken from a CT, MRI or other scanning procedures to obtainthree-dimensional scan data. Typically, the image scan data is placed ina computer to provide a three-dimensional database that may be variouslyused to provide graphic information. Essentially, such information isuseful in surgical procedures and enables viewing a patient's anatomy ina graphics display.

[0007] The use of stereotactic head frames is commonplace, for example,see U.S. Pat. No. 4,608,977 issued Sep. 2, 1986 and entitled, SystemUsing Computed Tomography as for Selective Body Treatment. Suchstructures employ a head fixation device typically with some form ofindexing to acquire referenced data representative of scan slicesthrough the head. The scan data so acquired is quantified relative tothe head frame to identify individual slices. Three-dimensional scandata has been employed to relate positions in a patient's anatomy toother structures so as to provide a composite graphics display. Forexample, a space pointer (analogous to a pencil) might be directed at apatient's anatomy and its position quantified relative to thestereotactic scan data. The space pointer might be oriented to point atan anatomical target and so displayed using computer graphicstechniques. Such apparatus has been proposed, using an articulated spacepointer with a mechanical linkage. In that regard, see an articleentitled “An Articulated Neurosurgical Navigational System Using MRI andCT Images,” IEEE Transactions on Biomedical Engineering, Volume 35, No.2, February 1988 (Kosugi, et al.) incorporated by reference herein.

[0008] Further to the above considerations, the need for relatingexternal treatment apparatus to a specific target arises in severalaspects. For example, the need arises in relation to the treatment ofinternal anatomical targets, specifically to position and maintain suchtargets with respect to a beam or isocenter of a linear accelerator(LINAC) X-ray treatment machine. Thus, a need exists for methods ofaligning beams, such as from a LINAC machine, to impact specifictargets.

[0009] Generally, in accordance herewith, an optical camera apparatusfunctions in cooperation with a LINAC machine and a computer to enabletreatment of a patient with a beam that is positioned and maintained ona specific target in a patient's body. In an embodiment, the camerasystem is located in a known position with regard to the LINAC machineand to detect index markers at specific locations on a patient's body.The markers employed during image scanning processes correlate toreference points for the scan data. Thus, by correlation, anatomicaltargets in the body, identified in the image scan data are effectivelypositioned with respect to the treatment beam from the LINAC machineidentified by camera data. Essentially, data accumulation,transformation and processing operations serve to correlate scan datawith camera data and thereby enable the desired positional relationshipsfor patient treatment as well as providing an effective graphicsdisplay.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In the drawings, which constitute a part of the specification,exemplary embodiments exhibiting various objectives and features hereofare set forth. Specifically:

[0011]FIG. 1 is a perspective and diagrammatic view of a compositesystem in accordance with the present invention shown with reference toa patient;

[0012]FIG. 2 is a perspective view of components somewhat similar tothose of FIG. 1 shown in more or less detail for further explanations;

[0013]FIGS. 3A, 3B and 3C are perspective views showing index markersset in accordance with the present invention;

[0014]FIG. 4 is a flow diagram describing a process in accordance withthe present invention shown in relation to a patient;

[0015]FIG. 5 is a side view showing localization apparatus in accordancewith the present invention and shown in relation to a patient;

[0016]FIG. 6 is a side view of another system for patient localizationgenerally in accordance with the system of FIG. 1;

[0017]FIG. 7 is a side view of an optical and ultrasound positioningsystem on a treatment machine in accordance with the present inventionshown in relation to a patient;

[0018]FIG. 8 is a perspective and diagrammatic view showing a videopositioning system in accordance with the present invention shown inrelation to a patient;

[0019]FIG. 9 is a series of views 9A, 9B and 9C illustrating video andgraphic reconstruction fusion in accordance with the present invention,shown in relation to a patient;

[0020]FIG. 10 is a perspective and diagrammatic view showing anapparatus for calibrating or aligning optical cameras with respect to atreatment machine in accordance with the present invention; and

[0021]FIG. 11 is a perspective view showing another embodiment of thepresent invention involving frameless stereotactic navigation on animage scanning machine apparatus and shown in relation to a patient.

DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENTS

[0022] The following embodiments illustrate and exemplify the presentinvention and concepts thereof, yet in that regard, they are deemed toafford the best embodiments for purposes of disclosure and to provide abasis for the claims herein which define the scope of the presentinvention.

[0023] Generally, the embodiment of FIG. 1 accomplishes optical locationand/or X-ray location of a patient's anatomy for treatment. A linearaccelerator (LINAC) X-ray radiation therapy machine, generally locatedat L (FIG. 1 upper left) provides a beam B (generally radiation with anisocenter) for treating a patient P (shown reclining on a platform orcouch F). Typically, the beam B has a principal axis that coincides at aspecific location (isocenter) and is positioned at a target in or on thepatient P.

[0024] Basically, scan data is stored to specify the location of atarget in a patient's body, generally defined in three-dimensional scanspace (as slice data) with respect to references. The scan data isstored in a treatment processing system T which receives further datafrom a camera system C. Specifically the camera system C senses theinstant position of the patient P and the beam B (in camera space) onthe basis of marker locations on the patient P and the machine L. Byusing similar or related reference locations, scan space and cameraspace are correlated and the data is transformed to a common coordinatespace. Accordingly, the beam B is related and displayed with respect tothe patient P. Consequently, the beam B can be positioned and maintainedto collimate at the desired target. Note that both the machine L and apatient-supporting couch F are moveable to accomplish and maintaindesired positional relationships between the beam B and the patient P asdescribed in greater detail below.

[0025] The LINAC machine L is mounted on a floor 3 and includes a gantry1 which rotates about a horizontal axis 2, the angle of rotation beingindicated by a double-ended arrow 2A. The gantry 1 is thus rotatablysupported on a floor-mounted column or support structure 4 whichincludes a panel 4A for direct manual control. Control also may beprovided from the treatment processing system T.

[0026] Remote from the support structure 4, the gantry 1 carries aradiation source R (shown in phantom) somewhat aligned with a collimator5 which shapes the X-ray radiation from the source R to pass generallyalong the axis as indicated for the beam B. Well known structures may beemployed as the radiation source R and the collimator 5. Specifically,the collimator 5 may be a multileaf, miniature multileaf, circularcollimator, cut block, or other type of X-ray aperture. Typical LINACmachines, as currently known, could be used for the LINAC machine Loperating to establish an isocenter point 7 (shown at the abdomen of thepatient P) which point 7 is the convergence point of the central X-rayradiation (beam B for representation) and lies on the axis 2 ofrotation.

[0027] As indicated above, the patient P lies on the couch F and isspecifically shown reclining on a couch top 11. The couch top 11 ismovable; that is it can be displaced in Cartesian translations asindicated by double-ended arrows X, Y and Z. Such displacements areaccomplished by a mechanism 10, shown in phantom within the couch F.Direct manual ctrol is afforded by the panel 4A with data ctrol from thetreatment processing system T. Note that the couch F also rotates abouta vertical axis 12 (mechanical linkage) as indicated by a double-endedarrow 12A. A third orthogal axis 14 (patient lateral) is indicated topass through the isocenter point 7 as will be described in detail below.

[0028] The camera system C comprises three cameras 17, 18 and 19 whichmay take the form of well known video cameras, infrared filteredcameras, linear CCD cameras (with either infrared or n-infraredsensitivity) or other cameras of acceptable design to resolve thecontents of a space. The cameras 17, 18 and 19 are fixed on a frame 6along with a light source 16, and are oriented to view the treatmentspace or field of the couch F, the gantry 1 and the patient P. Thus, thecamera system C senses the contents of a volume occupied by theabove-described elements. Additional cameras may be located in otherpositions within the treatment room viz. attached to the ceilings orwalls.

[0029] Recognizing that various forms of markers can be used, if theindex markers are of the reflecting type, a light source 16 (infrared)may be used to produce reflected light as indicated by dash line arrows26. Although the light source 16 may not always be necessary, it canenhance the signal-to-noise ratio of the reflected light from the indexmarkers as related to background. Note that for the same purposeadditional similar light sources could be fixed on the frame 6, forexample, near the cameras 17 and 19.

[0030] In operation, the camera system C senses several markers thatindicate specific reference locations or index points. Specifically, theindex points are sensed as indicated by markers 20, 21, 23 and 24 whichare on the patient P, located, for example, on or near the patient'sskin. As indicated, the markers 20, 21, 23 and 24 may take a variety offorms, for example, LED emitters, reflectors of light, reflectingspheres, reflecting dots or various other devices that can be tracked bythe camera system C in three-dimensional space. Also, radiopaque circlescan be adhered to the skin at points identified as by a tatoo or inkmark. Also, reflective markers can be placed precisely on tatoos or inkmarks on the patient's skin.

[0031] The index markers 20, 21, 23 and 24 provide references fortransforming image scan data (initially stored) to the coordinate spaceof the LINAC machine L, as sensed by the camera system C. That is,three-dimensional scan data taken during a preliminary procedure isstored in the treatment processing system T and is correlated to dataprovided from the camera system C by using common points as may bedefined by the markers 20, 21, 23 and 24. The combined data accommodatesthe desired position and orientation of the couch F and/or theorientation and collimation of the beam B to impact the desired targetin the subject patient P. The control operation is disclosed in greaterdetail below along with the correlation of data to provide a compositedisplay relating the patient P to the treatment structure of FIG. 1.

[0032] Regarding the camera system C, the individual optical cameras 17,18 and 19 essentially “look” at the position and orientation of thepatient P, that is, viewing the volume containing the patient P and theapparatus as explained above. The markers 20, 21, 23 and 24 can be“seen” by the cameras to track marker positions relative to theisocenter point 7 and the beam B. By way of a disclosing reference, seeU.S. Pat. No. 5,446,548, entitled “Patient Positioning and MonitoringSystem”, L. H. Garrick and S. F. El-Hakim, issued Aug. 29, 1995; as wellas an operating system identified as the OTS Optical Tracking Systemproduced by Radionics, Inc. of Burlington, Mass. or a Motion TrackingSystem available from Oxford Metronics, Oxford, England.

[0033] As indicated, the optical signal outputs from the cameras 17, 18and 19 are supplied to an optical image tracking processor 34 (FIG. 1,upper right) as well known in the field. In the operation of theprocessor 34, the individual camera data signals are translated intothree-dimensional position data (in the camera coordinate space) relatedto objects in the cameras' collective field of view and including theidentifying index markers 20, 21, 23 and 24. The resulting positionaldata defines the position of the patient P relative to objects in thefield of view of the camera system C (in camera coordinate space).

[0034] Another set of markers, 30, 31 and 32 are attached to the couchF, shown variously disposed on the couch top 11. The markers 30, 31 and32 also are detected by the camera system C to determine the orientationof the couch F relative to the camera system C. Thus, by using outputsfrom the camera system C, the processor 34 also provides data indicatingthe position of the couch F in camera space. Utilizing such data, theprocessor 34 functions with other components of the treatment processingsystem T to coordinate data and accomplish the functions as describedabove. Other components of the treatment processing system T include animager 35, a treatment and planning system 36, a /computer 37, acontroller 38 and an interface display unit 39. The detailed operationof the treatment processing system T is treated below.

[0035] Still another set of index markers 40, 41 and 42 is fixed on thegantry 1, also to indicate positions in camera space. Furthermore,markers 43 are fixed on the collimator 5 (end of the gantry 1)specifically to enable three-dimensional tracking of the gantry and thebeam B relative to the patient P and the couch F. Thus, the camerasystem C provides data to coordinate the treatment machine L, the beam Brelative to the patient P, aligning an anatomical target with the beam Bat the isocenter point 7, or other focus of radiation.

[0036] Recapitulating to some extent, it will be understood that asexplained above, during an initial procedure, scan data is taken fromthe patient, as by a CT or MRI scanner and stored in the imager 35. Inaccordance with one operating format, the scan data may comprise slicedata, three-dimensionally representing a portion of the patient P inscan data space. Of course, scan data space is distinct from camera dataspace, compatibility being attained by translating. to a commoncoordinate space. Transformations, using well-known techniques of theart, are accomplished by referencing certain markers, e.g., markers 20,21, 23 and 24 which are located on the patient P and identify referencepoints in both space coordinates.

[0037] As indicated, during the scanning process, the positions of theindex markers 20, 21, 23 and 24 on the patient P are determined in thecoordinate space of the scanner (CT or MRI, scan space) employed togenerate the image scan data. For example, for CT scanning, graphicreference markers can be radiopaque markers placed on the skin atpositions indicated by index markers 20, 21, 23, and 24. They could be,for example, radiopaque circles stuck down to the skin at points where atatoo or ink mark is made. Knowing the coordinates in the scan space,and the coordinate locations of anatomical objects relative to them(markers 20, 21, 23 and 24) the target sites to be radiated aredetermined relative to the index points defined by the markers 20, 21,23 and 24. As indicated, image scan data for the index-marked positionsis stored in the imager or image data storage memory 35 for use by theplanning system 36 and the computer 37.

[0038] In the treatment planning computer 36, positions are determinedfor the markers 20, 21, 23 and 24, relative to the anatomy of thepatient P and the selected internal anatomical targets. Accordingly,target coordinates within a volume are resolved in the scan datacoordinate system.

[0039] The specific locations of the points identified by the markers20, 21, 23 and 24 also are determined in camera space by the camerasystem C while the patient P is on the couch F. Thus, identicalreference locations are provided in the two coordinate systems (scan andcamera) enabling data transformations as well known in the computergraphics field. Specifically, the reference points are detected by thecamera system C. This can be accomplished, for example, by placing LED'sor reflective markers on the positions of the index markers as indicatedby tatoo marks or ink spots used during the image scanning as describedabove. The marker positions are thereby determined in three-dimensionalspace relative to the camera system. Further, the marker positions onthe patient's body are also determined relative to markers on the LINACitself such as 30, 31, 32 on the couch 11 or 40A, 40B, and 40C on thegantry 1. Data from the camera system C is provided from the processor34 to the comparator/computer 37 where the index marker locations arecompared to marker positions determined from imaging data to accomplisha “best fit” as well known. Accordingly, the image data defining thepatient is transformed to camera space. Thus, a target coordinate isdetermined from the treatment planning system 36 involving the explicitlocation of the target in relation to objects in the camera field ofview including the collimator 5 and accordingly the beam B.

[0040] The three-dimensional position of the isocenter point 7 (incamera space) of the LINAC L is determined and controlled from acalibration procedure as described below. Thus, an internal selectedtarget position as determined from the transformation into cameracoordinate space is determined relative to the couch F, the gantry 1,the beam B and the isocenter point 7. Such information is provided tothe controller 38 to position the gantry 1 and the couch F and thus, tocontrol the treatment. The display unit 39 then dynamically indicatesthe positional relationships with a graphic image.

[0041] Specifically, controller 38 controls the angles and shapes of theradiation beam B determined by the treatment planning system 36. Again,beam approaches also can be transformed via the comparator/computer 37to position the gantry collimator 7 and that of the couch F (actuated bythe controller 38). The controller 38 also can incorporate structure torecord and verify positional relationships including those of the beam Band the patient P as well as controlling the status of the beam B (onand off) and dose rates during treatment.

[0042] For an example of a standard controller 38 and treatment planningsystem 36 as may be used in the system, see the Mevatron Linac providedby Siemens Oncology Care Systems Inc., of Concord, Calif., as well asthe product, XKnife Treatment Planning System available from Radionics,Inc. A typical display of relevant information at each point in atreatment process is indicated by an interface and the image of thedisplay unit 39.

[0043] After determining the position of desired treatment target in thepatient P using the coordinate space of the camera system C and alsodetermining the relative position and distance of that target from theisocenter point 7, also in camera space, the couch F is moved to accessthe desired target with the isocenter point 7. In that configuration,the beam B is directed from the collimator 5 to the target. The processcan be automated, with appropriate sequencing by the controller 38 forcorrectively driving the couch F. Accordingly, the beam B is maintainedwith the isocenter point 7 on the desired target.

[0044] The camera system C can monitor the process as described aboveand provide a feedback signal for automatically driving the couch F.Beam positions and dose rates measured in monitor units through thecollimator 5 also can be preplanned and actuated either bymanual-operator control (panel 4A) or automatically through thecontroller 38.

[0045] If multiple targets or a broad target field are to be radiated,or if intensity modulation of beams is specified, the controller 38 canmove sequentially to different target positions within a generalizedtarget volume, for example, attaining sequential positions, defined inX, Y and Z coordinates as well as dose rates, all achieved to effect adesired pattern of radiation.

[0046] In a dynamic mode of the system, corrections may be provided forpatient movement during treatment along with continual confirmation ofthe patient's body position relative to the LINAC machine. If there isrespiratory body movement of the patient P, as would typically occur inthe torso region, the tidal movement can be observed by the camerasystem C tracking the index markers 20, 21, 23 and 24. Synchronizing theradiation from the LINAC machine L can assure that the anatomical targetis impacted by the beam 6 even though the patient's internal organs aremoving. This too can be controlled by the controller 38 with feedback tothe optical tracking processor 34 through the comparator 37.Consequently, the comparator 37 enables streamlining certain complexprocedures and even routine procedures, as compar d to standard currentradiotherapy steps relying primarily on laser lights associated with aradiation machine and tatoo markings on the patient.

[0047]FIG. 2 is a fragmentary view showing certain components of thesystem of FIG. 1, whole or fragmented and generally bearing similarreference numerals, however, modified in some regards to illustrateother embodiments of the present invention. Note generally that thecollimator 5 is representative of the LINAC machine L for treating thepatient P positioned on the surface or top 11 of the couch 10. Theentire processing system T1 of FIG. 2 may incorporate the elementsdescribed with reference to FIG. 1 and is embodied in a unit 39Aembodying a graphics display.

[0048] A camera system C1 (FIG. 2) comprises two cameras, 17 18, thatare stably secured (symbolically indicated), as to the ceiling of thetreatment room. The cameras 17 and 18 are fitted with annular rings 17Aand 18A, respectively, each containing a circular array of multiplelight sources 17B and 18B respectively. The light sources 17B and 18Bmay be LED's (light emitting diodes) to illuminate the markers on thepatient P and the LINAC machine L as symbolically represented by thecollimator 5A. The light from the sources 17B and 18B is reflected asindicated by dashed lines and arrows 17C and 18C extending from thesources 17B and 18B and reflecting back to cameras 17 and 18.

[0049] As illustrated a stereotactic immobilizer 42 receives the patientP and may take the form of an evacuated beam bag as commonly used inradiation therapy to immobilize the patient once a correct setup hasbeen established. Alternatively, the immobilizer 42 could be a preformedtray or alpha cradle to define a firm contour of the patient's body forrepeat positioning.

[0050] Index markers 20, 21, 23 and 24 are fixed on the patient aspreviously described with reference to FIG. 1. Recall that these markersidentify locations marked by radiopaque or MR detectable index markersfixed on the patient P at the time of the CT or MRI scanning. Thearrangement in FIG. 2 could be applied on the simulator couch top 11 tosimulate a preplan of the treatment setup or could be applied on thecouch for radiotherapy as for example a LINAC couch. The radiopaque orMR detectable index markers used during the CT or MR scanner phase canbe replaced in the arrangement of FIG. 2 by camera detectable indexmarkers placed at the same locations on the patient. In context of FIG.2, the camera system C1 determines the 3-D position of the index markerswith respect to the camera coordinate system, as discussed above.

[0051] In the context of FIG. 2, with the scan data recorded and theposition configurations being sensed by the camera system C1, a targetP1 within the patient P is established within a treatment volume P2. Thetarget P1 may be the nominal focus of radiation from the collimator 5A,and the contour of X-ray dose from the LINAC machine may be intended toengulf the target volume P2. In certain applications, it is desirable tomove the target to an isocenter 7 (FIG. 1) for convergence of radiationbeams to the target volume P2. Accordingly, as indicated above, thecouch 11 may be moved to accomplish the desired coincidence.

[0052] Also as noted, the terminal unit 39A incorporates the capabilityto control and display positional data. Specifically, as indicated, adisplay panel 39B indicates, in X, Y and Z coordinates, the position ofthe isocenter relative to a target in real time, e.g. currently, as wellas the angles C, G and A (corresponding to LINAC angles 12A for couchrotations, 2A for gantry rotations, and A for collimator rotations asindicated by the arrows in FIG. 1) regarding the beam 6 in thecoordinate system of the patient's anatomy in scan data space asrendered from the treatment planning computer embodied in the unit 39.

[0053] As described in detail above, the treatment couch 11 carriesindex markers 30, 31, and 32 which are tracked by the camera system C1to indicate the instant position of the couch 11 throughout a procedure.As the angles C, G and A are changed during treatment, the position ofthe planned anatomical targets P1 can be kept at the isocenter 7. Inthat regard, a feedback controller can be connected from the camerasystem C1 to the treatment processing system T1 to automatically lock-inthe target with the isocenter. For example, the operation could involvean automated and integrated process of frameless optical tracking toaccomplish the desired treatment planning parameters and LINAC machinecontrols for patient positioning.

[0054]FIG. 2 also shows alternative types of index markers, for example,marker 50 has a machine recognizable geometric pattern detectable by thecamera system C1 to determine the orientation and positioning of thecouch top 11. Such markers may take the form of bar-graph patterns,geometric shapes (e.g. triangles), lines, two-dimensional geometricfigures and so on, any of which can be detected by the camera system C1with positions determined by the treatment processing system T1. Thedetecting and processing of such geometric shapes is well known in thefield of optical tracking technology and accordingly it is noteworthythat the discreet index points, as indicated by markers 30, 31 and 32 onthe couch top 11 may be replaced by geometric patterns. Also note thatindex markers 51, 52 and 53 are fixed on the immobilization tray 42.They may be LED's, reflective spherical surfaces, used as augmentationalredundancy of the index markers on the patient's body and/or the couchtop.

[0055] A plate structure 55 illustrates another alternative geometricshape specifically including a triangular plate carrying a plurality ofraised spheres 56 along with a linear stripe 57. The plate 55 may beadhered to the patient P indexed by tatoos or other marks. For example,a line 58 may be drawn on the patient P during the CT scan process as areference. In summary, note that the structure of the plate 55 providesconsiderable character for indicating the orientation of a patient'sbody.

[0056] Still another form of indicator or marker is exemplified by astrip 60 of reflective tape adhesively secured to the patient P. Again,such a marker can be used as a reference relating to the scan data. Notethat by using a comparator algorithm to compare curvilinear geometricobjects between the imaging (scan data collection) procedure and thetreatment phase (camera space) an indication of the patient's bodyorientation can be determined and the coordination of target positionsmanaged.

[0057]FIGS. 3A, 3B and 3C show other exemplary forms of markers asgenerally introduced in FIG. 2 that are useable for tracking inaccordance with the present invention. FIG. 3A shows a tatoo 60 whichmay have been made on a patient's skin preparatory for CT scanning. Theindicated location would correspond, for example, to the desiredplacement position for a radiopaque apertured disk or marker detectableduring the scanning. Later, preparatory to treatment, a retroreflectiveapertured disk 61 is applied to the patient precisely as indicated bythe tatoo 60. An aperture or hole 61A is defined in the center of thedisk 61 for registration with the tatoo 60. In an alternative form, thedisk 61 may define a reflective dome or spherical surface of areflective nature for effective camera detection.

[0058] In FIG. 3B, a geometric reflective plate 62 of triangularconfiguration is adhesively secured to the patient P functioningsomewhat similar to the plate 55 as considered with reference to FIG. 2.Plate 62 defines holes 63 and 64 to enable precise placement withreference to marked locations on the skin of the patient P.

[0059] Another alternative form of marker is shown in FIG. 3 andincludes an array of spaced-apart, reflecting spheres 66A, 66B, and 66Cfixed to a shaft or stock 65 defining a threaded distal tip 67. In use,the marker is threadable engaged with bone B beneath the skin of thepatient P. An example of the marker's use would be to d termine theorientation repeatedly of a pelvis location for prostate orgynecological irradiation. Such markers could be percutaneously fixedinto the iliac crest bone of the pelvis at one or more locations andremain there for a duration of treatment. The marker also could be putin at the time of image scanning to produce scan data. The array ofspheres could then be attached to a stud section emerging from thepatient P, for example, at the time of treatment to provide a reflectivesurface. Clusters or triads of reflecting spheres or other geometricobjects or shapes could be attached to one threaded shank adapter toprovide both position and orientation information with respect to thepelvis. The spheres could be attached and removed repeatedly from theshank for repeated relocation.

[0060] Note generally that retro reflective material as may be used inthe various markers as described herein is well known, having acharacteristic to reflect illumination substantially back in thereceived direction. Bright, shiny, or colored surfaces may bealternately used to suit the camera detection needs or discriminate onemark from another. Such surfaces are particularly useful in someapplications hereof.

[0061] Further with respect to the use of markers as disclosed herein,markers in the form of geometric objects may be attached to indicatepositions according to the needs of the various procedures includingimage scanning, simulator planning and treatment. The patient locationssuch as the lateral or anterior portions of the skin that are visible tothe camera are often advantageous. Orientation of detectable plates,sphere, disks, domes and so on can be determined based on viewing anglesof a camera system for optical visibility. Incidently, markers withlinear patterns coincident with the alignment of lasers or otherfiducials could be advantageous in exemplifying the setup and relocationof a patient on a treatment couch.

[0062] Referring now to FIG. 4, consider a process involving the systemsof FIGS. 1 and 2. An initial step, illustrated by block 70, is scanningthe patient by CT, MR, X-ray ultrasound, PET, or any other modality orby the use of simulators to obtain three dimensional data. A simulatoris an X-ray or CT scanning device which has a couch similar to that ofFIG. 1, in which X-ray or tomographic image data enables a clinician toestablish targets within the body relative to external or internalanatomical landmarks. Image data and information on desired targets areachieved as illustrated by the block 71 (FIG. 3). Such data can be takenwith fiducial markers, as described above and in parent cases, toregister the data in scanner or stereotactic coordinates. This data isinputted to a treatment planning computer (e.g. system 36, FIG. 1) toestablish the treatment plan illustrated by block 72 (FIG. 4). Targetposition data, along with target volume and beam position data aredetermined by the clinician in accordance with clinical needs.

[0063] After the treatment planning, the patient is put on the couch Fwith an appropriate setup as illustrated by the step of block 73.Alternatively, during the step of block 73, the patient could be placedon a diagnostic apparatus such as an interoperative CT or MRI scanner.By use of an optical tracking system, as described above, furtherreference data is taken on the treatment machine, e.g., machine L(FIG. 1) in a step illustrated by block 74 (FIG. 4). Also within thestep, a transformation can be made via a computer or comparator (e.g.,comparator 37, FIG. 1) to establish the position of treatment plantargets relative to the coordinate space of the camera system.

[0064] Next, the distance or difference in position of the plannedtarget from the LINAC (isocenter point 7, FIG. 1) is established and thepatient P is moved. to align the target or targets with the isocenter ofthe beam B. The step is illustrated by the block 75 (FIG. 4).Furthermore, the beam positions and shapes of the collimator (collimator5, FIG. 1) can be established and also set on the LINAC machine L asindicated by block 76 (FIG. 4).

[0065] Further refinement of internal target positioning to an isocentercan be achieved by X-ray imaging. As an example of this, referring toFIG. 1, X-ray machine components 80 and 81 are aligned to the axes 14(horizontal) and 12 (vertical), respectively, and X-ray screen 84 forX-ray machine 80 can thereby determine a digital image of X-rays throughthe patient's body. A similar screen (not shown) functions with theX-ray machine 81. Further, a portal imager 85 (a common device on modernLINACs) can provide a digital image from the high energy X-rays emittedfrom collimator 5. Thus, diagnostic X-rays from machines 80 and 81 orhigh energy X-rays for portal imaging can be used to visualize internalanatomy such as bones and/or radiopaque index markers placed on the skinor implanted in bones or tissue within the patient prior to treatment.

[0066] Once the patient position translations described above (based onexternal landmarks) have been done, then the internal anatomy, which maybe more closely represented by, for example, the bony structures withinthe body, can be further used to verniate and/or qualify the position ofa desired internal target to isocenter. For this purpose, the treatmentplanning computer could provide simulated or reconstructed port filmviews or digital reconstructed radiograms (DRR's) to simulate such highenergy X-ray or diagnostic X-ray images through the patient. These arecompared by overlay analysis, image fusion, or other computer theoreticcomparative methods to the actual port films or X-ray shots, asillustrated by block 84 of FIG. 4. Based on the comparative images fromsuch reconstructed and actual X-ray views, further incrementation of theX,Y,Z movement of the couch can be made or planned. This is actuated asillustrated by step 85. Again it could be done automatically with afeedback system for fast image fusion comparison of simulated X-rayviews.

[0067] Another embodiment of the present invention could include adiagnostic apparatus. For example, it may be desired to locate a patientin an CT, MRI, simulator, X-ray, PET, or other imaging machine in ananalogous way to the example above of positioning a patient in a LINAC.For an interoperative CT or MRI scanner, it may be needed to move atarget from one historic image scan episode to the scan slice plane(s)of the interoperative image scanner to determine the degree of residualtumor during operative resection. Thus the present invention includesuse of diagnostic apparatus substituted in the examples given, forexample LINACs.

[0068] Referring to FIG. 5, an embodiment of the present invention isillustrated for use in cranial, head and neck, torso, or pelvisapplication. The cranium of the patient P is stabilized by an armstructure 86 (left) which has attached index markers 87 and 88 fordetection by a camera system C2. Various index markers on the patient'shead 89, chin 90, throat 91, upper torso 92, and torso 93 areillustratively shown, depending on the clinical application and regionto be treated. These indicate the orientation of the patient's anatomy,and enable a comparison of that orientation to the position of thepatient during the image scanning phase. As explained above, these indexmarks could be in the same location as image visible index markersplaced on the body during the scanning phase. Alternatively, the indexmarkers could be randomly located or located in position to suit thetreatment setup. In that case, the registration from camera space toimage scan space can be done by surface fitting, best matching of indexpoints to surface contours, or other similar procedures utilizing indexmarker positions and surface contours from scan data and camera data.

[0069] As shown in FIG. 5, the LINAC or treatment couch 11 has indexmarkers 31, 32, and possibly more not shown. To help orient the torsotranslations and angulations locally in addition to facilitatingpossible couch movements, a so-called “tectonic plate” 100 is placedunder the patient P. This can be moved in the plane of the couch top 11,as described in a parent application. It can also provide elevationmovements which are accomplished by an inflated cushion 102 between anupper plate 101 and a lower plate 100. Inflation of the cushion can beactuated by an inflater 103, which could be manual or electronic. Fineverniations of the height of the torso relative to the head, forexample, can thereby be achieved. Monitoring of the position of thetorso relative to the head could be done by the camera system C2 bynoting the 3D position of such index markers as marker 92 compared tomarkers on the cranium such as markers 89 and 90.

[0070] An alternative means of determining the orientation relative tothe LINAC of the pelvis or other portion of the body is achieved by abelt structure 104 which can be placed on the pelvis repeatedly in asimilar position. This may be achieved by sticking the belt 104 on orattaching the belt along an index line such as line 105 which is markedby pen on the patient's skin at the time of scanning or simulatorplanning. The belt 104 may have a multiplicity of physical markers suchas marker 106 so that the camera system C2 can determine the orientationof the belt 104 and thus the orientation of the pelvic region relativeto the LINAC couch and relative to the isocenter of the LINAC. In thisway internal targets such as the target point 107 (in the neck) or atarget point in the pelvic region such as at the prostate or cervix 108could be “driven” or moved to the isocenter position illustrated bypoint 109 by means of X,Y,Z translations of the couch 11, as describedabove. Also shown in FIG. 5 is a schematic representation of thecollimator 5 with its index tracking markers 43A, etc. so thatcorrelation of beam and bodily positions can be tracked by cameras 16 ofthe camera system C2.

[0071] Referring to FIG. 6, another embodiment of the present inventionis shown wherein natural surface contours of the body are fused withreconstructed contours to position the patient P on the LINAC couch top11. A camera system C3 can be a video camera system to visualize theactual visual scene of the patient P on the couch top 11 and LINACmachine represented by the collimator 5. In this case, the cameras maybe unfiltered, two-dimensional CCD cameras which have been calibratedfor stereoscopic viewing. Two, three, or more cameras can be used. Somecan be filtered for infrared reflective viewing and others could beunfiltered for direct video imaging. They can be mounted on the ceilingof the LINAC room (fixation not shown). Alternatively, the cameras ofthe system C3 could be individual and separated, each located forexample on the walls or ceiling of the LINAC room.

[0072] An illumination system 115 also is represented which projects agrid of light onto the patient P, illustrated by lines of a surface 117.This could be a pattern of structured light with areas of light and darkand linear light arrays in two dimensions projected onto the patient'sbody surface. Such a light array can be recognized and registered bypattern recognition algorithms in a video scene. The VISLAN systemdeveloped by A. Colchester illustrates methods of such surfacereconstruction, as disclosed in an article “Development and PreliminaryEvaluation of VISLAN, A Surgical Planning and Guidance System Array WithOperative Video Imaging”; A. C. F. Colchester, et al., Medical ImageAnalysis, Vol. 1, pp 1-18, Oxford University Press, 1996.

[0073] Information from camera system C3 is represented by signalsapplied to a video processor 112 to capture the video field of view andto reduce the locus of points of structured light on the surface 117 toa set of three-dimensional points in space relative to camera coordinate118. Thus a rendering of a portion of the surface of the patient's bodycan thereby be done. The cast light could be by laser or patternprojection and could be in different frequency ranges (visible orinfrared) as different colors and patterns to better differentiatepatterns and backgrounds.

[0074] Image scan data, supplied by a data computer represented by ablock 35, also can be segmented to render the reconstructed surface ofthe skin of the patient P. See by reference the XKnife System ofRadionics, Inc., Burlington, Mass. This would provide an analogouscomputer graphic rendering of the same surface information as in thevideo processor 112. Those two surface data sets can be input to animage fusion computer 114 which implements an image fusion algorithm tofuse the video surface and the reconstructed image base surfacesdescribed above. This can be done by a chamfer algorithm, an example ofwhich is embodied in the Image Fusion algorithm of Radionics, Inc.,Burlington, Mass. Such an image fusion of surfaces provides aregistration of the 3D data set from the image scan to the coordinatesystem of the video processor. This is a transformation from thestereotactic image data set of the image scanner to the 3D coordinatesystem of the camera space 16. Since the camera is also registeredrelative to the external LINAC apparatus, its couch, gantry, andcollimator, this provides a transformation of the image data set to thecoordinate space of the LINAC.

[0075] As illustrated in FIG. 6, in the process of treatment planning, atarget position 44 and target volume 45 are determined in the body andrendered in the image scan data of the computer 35. The coordinates ofthese structures in turn are transformed as just described to thecoordinate system of the camera space. Therefore, the position of thetarget point 44 in the camera space is “known” by the camera system andits associated optical processing and computer storage processor 112.

[0076] The output from the video processor 112 and the image data plustreatment planning data from the imager 35 enter the image fusioncomputer 114. After image fusion of the reconstructed image data surfaceand the video detected surface, the target coordinates and target volumeinformation from the computer 114 are sent to the LINAC controllercontrols 38. This will enable either manual positioning of theanatomical target 44 to the LINAC isocenter point 7 or actuate automaticcontrols to do the same. The user interface and display system 39 nablesthe clinician to assimilate all of this information visually and toactuate the movement of the couch 11 for the translation just described.These movements are indicated by two of the coordinates, Y and Z in FIG.6.

[0077] Also shown on the couch 11 are various geometrically detectableindex structures 120 and 122, which can be detected by the video camerasystem C3 and their position determined in 3D space. This will monitorand control the position of the couch 11 and control the movement of thecouch during corrective changes. An immobilization cushion 121 is alsoshown which can help in certain clinical situations to prevent movementof the patient.

[0078] Also shown in FIG. 6 is a portal imaging system 85. Suchportal-imaging digitized detectors are common on commercially availableLINACs today. A beam from collimator 5 (representing the LINAC) is sentgenerally in the direction of the principal axis 6 through the patient'sanatomy and passing by isocenter point 7. Bony structures within thepatient's anatomy will be seen on the digital portal image. Once thepatient's body has been moved to the desired position by the videotracking described above, such a portal image can be taken at particulargantry, couch, and beam positions. From the 3D image data, areconstructed projected portal image to render the skeletal detailsinside the body can also be generated to simulate the same direction ofthe beam in physical space. A correlation or difference in positioningof the portal image compared to the reconstructed portal image will alsogive information on translation and rotation corrections for the patientpositioning on the couch 11 with respect to the LINAC machine(collimator 5) so as to bring these two portal image views into closerregistration. This can give incremental values of X,Y, and Z to furtherverniate the desired target spot to the isocenter. By reference, notethe article entitled “Automatic On-Line Inspection of Patient Set-Up inRadiation Therapy Using Digital Portal Images,” by Gulhuijs, K. G. A.and vanHerk, M., Med. Phys., 20(3), May/June 1993.

[0079] Also shown in FIG. 6 is the portal imaging processing electronicsand computer indicated as a block 124. This processor develops data.from the portal image detector 85 to render two-dimensional projectedviews through the patient's anatomy. This data, with image information,is then supplied to the image fusion computer 114 to enable imagecorrelation with respect to reconstructed portal images from the imagedata computer 35. Image fusion computation in the computer 114 therebyderives LINAC control parameters which are sent on to block 38 toactuate patient verniated movement.

[0080] Referring to FIG. 7, another embodiment in accordance with thepresent invention is shown to provide target and patient positioning. Anultrasonic detector 130 (center) creates ultrasonic image data within animage field indicated by dashed lines 133A and 133B. Within that fieldan image of internal anatomy is detected and processed by an associatedultrasonic processor 135. This can include a display of the actualimage. Such ultrasonic images are commonly used clinically, for examplein equipment made by Aloka Corporation of Wallingford, Conn.

[0081] Index markers 131A, 131B, and 131C are attached to the ultrasonicscanner 130 so that camera system C4 can detect in three dimensions theorientation of the ultrasonic unit relative to the patient P. Otherindex markers may be placed on the patient's body such as marker 20 forpurposes of registration of the body anatomy as well. Thereby a targetpoint 44 can be identified, and because its position is known in thecoordinate space of the ultrasonic imager 130, and because the positionof the ultrasonic imager 130 is known in the coordinate space of thecamera 16, then the position of target point 44 can be known byappropriate transformation in the coordinate space of the camera C4.

[0082] A target volume 45 also can be detected by the ultrasonicdetector 130. Its 3D position may also be thereby determined in the 3Dcoordinate space of the camera system C4. This, then, illustrates anexample of a real-time image scanner to provide updated positioning ofinternal organs and tumors. Use in soft tissues such as prostate,breast, head and neck, larynx, liver, and so on can enable correctionsto organ shift that may occur from initial CT, MR, or other scanning.Computer 136 can compare or image fuse current ultrasound images fromthe processor 135 to historic scan data and/or camera position data tomake body position corrections. Position corrections and interfacedisplay by LINAC controls 38 and display 39 are similar to the examplesgiven previously to move target 44 to isocenter 7 of beam 6 of LINAC.collimator 5. A similar example to this could substitute aninteroperative CT or MR scanner for the ultrasonic image, with opticalindex markers analogously attached to the CT or MR interoperativescanner.

[0083] Referring to FIG. 8, another embodiment in accordance with thepresent invention illustrates the use of multiple video cameras toreposition the body on a radiation treatment or simulator couch. Cameras140A, 140B, 140C, and 140D view the patient's body from a variety oforientations. More or less numbers of video cameras could be present inthis embodiment. In a particular arrangement, cameras 140B and 140D arecolinear and opposed, viewing along a central axis 142. Camera 140Aviews along a separate principal axis 143, which may be orthogonal tothe axis 142. Camera 140C may be viewing from an oblique axis 144. Axes142, 143, and 144 may be prealigned to intersect at a point 141. Forexample, the point 141 may be precalibrated to be the LINAC isocenter.

[0084] The collimator 5 has a central axis 6 (beam) which also may passthrough the point 141 as the isocenter of the radiation beam as well asthe camera views. It is not necessary that all the camera axes havecoincident axes. They may be set at arbitrary directions and calibratedto the scanner or treatment machine coordinate space in a mannerdescribed in connection with FIG. 10 as described below. Byprecalibration, the position of the isocenter 141 may be known virtuallyin the camera coordinate space of each of the cameras and in each of thecamera views. This may be convenient, depending on clinical setting andpatient and treatment setup. One of the cameras also may be tracking theposition of the couch 11 and another camera may track the collimator 5geometry and specifications of the LINAC space and room. The cameras mayhave a known calibration in the 3D space of the room. An example of acalibration procedure and system is shown below.

[0085] Also shown in FIG. 8 are index mark positions 20, 21, 23, 145,146, and index line 60. Similar to the description above, these may beradiopaque or MR visible markers which can be “seen” in the image scandata. Their position may be referenced on the body by ink marks,tattoos, or lines which are visible by video cameras 140A, 140B, 140C,and 140D. Index markers 20, 21, and 23 may be discrete or geometricobjects similar to those described above placed at positions on theupper or anterior surface of the body. Markers 145 and 146 may bemultiple markers on the lateral portion of the body. Similarly,geometric objects such as stripes, triangles, or recognizable patternsof lines or shapes, illustrated here by the example of linear objects60, could be similarly placed so that they are visible to one or more ofthe cameras at the same time. These can be used as described below toprovide reference points to correlate real video images of the body toreconstructed video representations or simulations of the body based onimage scan data.

[0086] The electronic signal output from the cameras 140 may beprocessed by video electronics, indicated by the processor of block 34in FIG. 8. The processor 34 provides power and strobe signals to thevideo cameras. Output data from the video cameras generates electronicsignals for a display unit 150 which includes a comparator, displaysoftware and a display device, such as a CRT. Real video views of thepatient's body on the treatment couch top 11 can be reduced to digitaldisplays in a calibrated relationship in terms of their magnification,relationship to the isocenter point 141, and relationship to otherpoints in the 3D space of the treatment/diagnostic room.

[0087] The block 35 in FIG. 8 provides the image scan data taken fromCT, MR, ultrasound, X-ray, PET, simulator, or other modalities. Thisdata is input into a planning computer 36 and used to determine targets,beams, etc., as described above. The external anatomy of the patient'sbody, i.e. the skin, can be rendered as a 3D surface in the space of theimage data by the computer 36 (see for example the XKnife planningsystem of Radionics, Inc., Burlington, Mass.). The image scan data canalso include both locations of the mark points 20, 21, 23, 145, 146, ormark objects such as 60 by use of appropriate scan-visible scanner indexmarkers placed at these positions during image scanning. Also, projectedviews or simulated reconstructed views of such 3D surface renderings canbe developed by planning computer 36 to simulate video views from anydirection. Similarly, projected positions of the scanner index markersonto the 2D reconstructed views for each video camera can be developedin computer 36. Such reconstructed video views in the directions of axes142, 143, and 144 are created by computer 36 based on the image scandata in image scan coordinates.

[0088] Selected target point(s) such as 44 or a target volume 45 arecontoured and segmented by the clinician in computer 36. The projected2D reconstructed video views, including projected target positions fortarget 44 and volume 45 from the 3D image data can be input into acomparator system 150, which may be the same computer 36 or a separatecomputer with graphic display means. Thus, in the comparator computer150 input data from the real video views and data from reconstructedvideo views can be compared, merged, image fused, or renderedcontemporaneously. In this way, the position of the target point 44 orvolume 45 from the image scan space may be seen relative to thecoordinate space of the camera views. Also, the projected view ofisocenter 141 can be displayed in each video view so that the operatorcan determine the couch or patient translation(s) within each of theviews to bring the selected target point 44 into coincidence withisocenter point 141. Such translations can be represented as output fromthe comparator system 150 to, for example, the LINAC or diagnosticcontrol system 38. The LINAC/scanner controls can provide signals to thecouch motor system 151 to enable X, Y, and Z translation of the couch soas to move target 44 into physical coincidence with X-ray beam or imagerisocenter 141. When so done, the X-ray beams from collimator 5 willconverge on the isocenter and therefore the target point. For a LINAC,dosimetry from the planning computer 36 may be delivered by means of theappropriate orientation and collimator shape out of the LINAC collimator5. Control of the couch position, gantry movement, beam configuration(for example a multileaf collimator or shaped beam collimator), as wellas data to record and verify system can be output from the LINAC controlsystem 38. The process of patient positioning, monitoring, positionfeedback, dose delivery, and angulation of the beams can be carried outmanually or by automatic control.

[0089] Referring to FIG. 9, exemplary images are shown that may berendered from the comparator computer and software and display means150. These may be views on a computer graphics screen, CRT, liquidcrystal display, or other display means or alternatively graphicallyoutput by printout. In FIG. 9A, the solid line 154 represents theprojected outline of the patient's torso as viewed, for example, bycamera 140A in FIG. 8. It may represent the direct video image of thepatient's body on couch 11. It may be enhanced by appropriateillumination, structured light scanning, laser beam wash over thesurface, infrared lighting, or just natural lighting. Point 158 mayrepresent the position of the beam isocenter 141 as projected into theview plane of camera 140A. The cameras may be precalibrated prior to thesetup so that the projected position of isocenter point 158 can becalibrated within this field of view of the camera 140A.

[0090] The dashed line 155 represents the boundary of the externalcontour of.the body from the projected reconstructed view derived fromthe prior image scan data along an axis parallel to axis 143. Dashedlines 155 then represent a computer generated contour of the externalprojection of the patient's body to simulate the actual video boundaryline 154. The non-coincidence of dashed line 155 compared to solid line154 in FIG. 9A represents the degree of translational shift or bodymovement needed to bring the lines into registration. Projected targetposition 156 and volume outline 157 are shown in the reconstructed videoviews based on imaging data.

[0091] Also shown in FIG. 9A are the positions 220, 221, 223, and 260 ofdiscrete geometric optical index markers detectable by camera 140A thatare located in the positions corresponding to markers 20, 21, 23, and 60in FIG. 8. These can be the position of discrete geometric scanner indexmarkers placed on the body during the scanning phase and image datacollection. In the reconstructed view of the image scan data accordingto the direction of camera 140A, positions of objects 230, 231, 233, and270 correspond to the reconstructed projected views of the scanner indexmarkers, as seen in the image data. For correct alignment of thereconstructed image scan projections to the actual video projections,the markers 230, 231, 233, and 270 should coincide in the cameracoordinate space to camera marker coordinates corresponding to theoptical index markers 220, 221, 223, and 260.

[0092]FIG. 9B illustrates the result of a computational translation ofthe dashed line 155 to coincide with the solid line 154 from FIG. 9A. InFIG. 9B, the dashed line 155A (which is the translated and/or rotatedanalog of external contour line 154 in FIG. 9A) is now lying close tothe solid video image outline of the external surface 154. Bringing thetwo lines 154A and 155A into coincidence can be done manually by theoperator by manipulation of the display in 150 or it can be doneautomatically by a mathematical algorithm in 150 which recognizes thetwo lines and image fuses th m by a lin minimization approximation,chamfer algorithm, or curve fitting process. This would give rise,therefore, to a virtual positioning of the selected target point 156Aand volume outline 157A with respect to the actual video projection line154. With this registration having been done, then the associatedtranslation shifts ΔX and ΔZ, as shown in FIG. 9B, can be determinedfrom the display or the computer output of 150. Thus ΔX and ΔZcorrespond to the translations of the couch 11 in FIG. 8 required tobring the selected target point 156A into coincidence with the isocenterpoint 158 as viewed in the projection parallel to axis 143. In thisexample, the patient is lying substantially horizontal on the couch top11 in a similar position to the orientation of the patient on a CTcouch, for example, where a horizontal is established. Otherwise, asequence of rotations and translations can be implemented mathematicallyfor a similar coincidence-of target point to isocenter point formultiple camera views.

[0093] In the situation that non-natural scanner index markers are used,such as elements 20, 21, 23, and 60 in FIG. 8, it may be convenient touse the camera marker coordinates in the 2D projected views for theseelements, as shown in FIG. 9A, to produce the translation and/orrotation of the patient's body so that the video image and thereconstructed video image (from the image data) coincide. Shown in FIG.9B is the resultant coincidence of reconstructed scanner markercoordinates as projected into the video camera views with the cameramarker coordinates from the optical index markers detected by thecameras themselves. Here the translation and/or rotation of the body issuch that the camera marker coordinates 220, 221, 223, and 260 coincidewith the reconstructed positions of the scanner index markers 230A,231A, 233A, and 270A. Use of such geometric objects could have certainadvantages when illumination levels and circumstances make difficult thevisualization of the external borders of the patient's anatomy for theimage fusion to the reconstructed external borders, as described above.Either one or the other method may be used and advantageous according toa given clinical situation.

[0094] Referring to FIG. 9C, a projected view of video surface contour160 as seen from video camera 140B is brought into coincidence with areconstructed video view from direction 142 as determined in treatmentplanning computer 36. The external contour of the patient's body isindicated by the dashed line 161. The appropriate mathematical shiftingof the treatment planning external contour has been done in 150 so as tobring these projected surface contours into coincidence, as discussed inconnection with FIGS. 9A and 9B. Furthermore, the target position 162and treatment volume 164 can be rendered in the projected 2D view of the3D data from the image scanning, and these also are shown in FIG. 9C inrelation to the real video contour 160. The component distances ΔX andΔZ similarly correspond to the couch translations to make the targetpoint 162 coincide with projected isocenter point 159.

[0095] As an alternative, or in addition, also shown in FIG. 9C are theoptical index markers 245 and 246 corresponding to scanner index markersplaced on the locations 145 and 146 shown in FIG. 8. The scanner markercoordinates for these scanner index markers can be developed in theimage scan data, as described above, and rendered from the dataprocessing or treatment planning computer as reconstructed scannermarker coordinates or sets of coordinates, as illustrated by the circles255 and 256, shown in coincidence in FIG. 9C with the optical indexmarker positions 245 and 246. It can be that for the various views ofcameras 140A, 140B, 140C, and 140D of the example in FIG. 8, thatlocation of such optical index markers corresponding to scanner indexmarker positions can be placed conveniently on the frontal, lateral, oroblique surfaces of a patient's anatomy for this purpose.

[0096] It is noted that in some circumstances such scanner index markersand optical index marker positions may be convenient for real-time videorepositioning of a patient's body, as illustrated in the example of FIG.8 and FIG. 9. This may be an alternative to or an augmentation of apurely external contour or 2D surface contour projection or a 3D surfacecontour matching of natural anatomical landmarks.

[0097] The example of FIGS. 8 and 9 illustrates an apparatus and methodwhich is in accordance with the present invention that does not requirepredetermined fiducial markers to be placed on the external anatomy oruse of structured light illumination. In the situation where no scannerindex markers are used, the system and method of the present inventioncan rely on natural landmarks such as surface contours or edges ofexternal body surfaces to be brought into registration in a virtual viewof image data compared to an actual video view of the real scene. Theincrease in the number of cameras from many view angles such as camera140C at an oblique viewing angle 144 increases the input data on thereal external surface. The corresponding matching or surface fusion ofthe reconstructed surface from image scan data to data on the surfacefrom multiple camera views will improve with the increase in cameranumber and views. The number of cameras and the degree of suchregistration may depend on the clinical circumstances and the particularbody region that is being treated. Such registration could haveapplication in the cranial, head and neck, torso, abdominal, and pelvic,or even limb extremity for treatment using external beam irradiation orfor diagnostics using a CT, MRI, or other scanner type. In thisconnection, reference is made to use of video cameras in a treatmentplanning room in the paper by B. D. Milliken, et al., entitled“Performance of a Video-Image-Subtraction Based Patient PositionerSystem,” Int. J. Radiation Oncology Biol. Phys., Vol. 38, pp. 855-866,1997.

[0098] Referring to FIG. 10, apparatus is shown for calibrating a camerasystem to the isocenter position and principal axes of a treatmentplanning machine, image scanner, or simulator. Camera system C4 ispositioned to view the treatment or imaging field. The lasers 160, 161,and 162 are positioned to send laser beams 160A, 161A, 162A to convergeat a common point. This point, for example, may be the isocenter of aLINAC. Alternatively, the lasers could cast sheets of light in planeswhich include the isocenter. At the isocenter is placed a marker object170, which may be a source of light, a globe-emitting light, an LEDlight source, a retroreflecting, sphere, a reflecting geometric object,an object with a specific geometric pattern of lines, crosses, diamonds,other objects, and so on which would indicate the position of theintersection of the laser beams and therefore the position of isocenter.Camera system C4 detects the field including the object 170. Since thiscan be registered in the output data from the video cameras, which isprocessed by a CCD camera or video camera processing electronics andcomputer 177, then the electronic data corresponding to the 3D positionof the object 170 is thereby determined. The camera processor 177 canstore that position, and when 170 is taken away and a patient put inplace, then 177 can refer to all other 3D points in space with referenceto it. In this way, camera system 16 is calibrated with respect to its3D coordinate space and in respect to the point corresponding toisocenter where the object 170 is placed. The object 170 could bepre-aligned and calibrated with the laser beams 160A, 161A, 162A by aseries of light detection measurements prior to camera calibration. (Byreference, see the MIS Mechanical Isocenter Standard of the XKnifeSystem by Radionics, Inc., Burlington, Mass.).

[0099] Also shown in FIG. 10 are video cameras 140A, 140B, and 140D,which are analogous to those used in the embodiment of FIG. 8 inaccordance with the present invention. These could be an alternative oran augmentation of the camera system 16 according to the clinical needs.The cameras 140A, 140B, and 140D are shown in this example colinear withthe lasers 160, 162, and 161 only for the purpose of illustration.Indeed, the video cameras and the lasers may be very close together orthe laser beams may be delivered colinearly with the cameras by means ofsplit prisms or beam-splitting mirrors so that the lasers themselves donot obstruct the camera view. The calibration structure 174 may haveadditional markers visible on lateral views such as 172, 173, and 175 togive a perspective and magnification calibration for the lateral cameras140B and 140D. The video cameras 140A, 140B, and 140D may be used forrepositioning external contours of the patient or may be used to developvideo data of optical index markers to produce camera marker coordinatesin accordance with the discussion above. With three or more non-colinearpoints in any camera projection, perspective use of the cameras can bedeveloped whereby calibration of the cameras relative to, for example,the isocenter of a linear accelerator could be made and embedded in thepositioning computer 178 in FIG. 10.

[0100] Also, to calibrate the laser axes in the coordinate space of thecameras, other objects such as 171, 172, and 173 are placed in knownpositions relative to these axes, and also detected by camera system C4.Again the camera processor 177 can record this data and determine in itsstereoscopic 3D coordinate frame the position of the axis points 171,172, and 173 as well as the origin point 170. In this way, the 3Dcoordinate system associated with imaging scanning, simulator, ortreatment machine can be calibrated and transformed into the 3Dcoordinate system of the camera 16.

[0101] A processing computer 178 may also be connected to the cameraprocessor 177 for the purpose of storing such spatial information andfor the purpose of registering other 3D points which may come into thefield of view of the cameras relative to the transformed coordinatesystem as described above. When a patient is placed on a LINAC treatmenttable with a calibrated camera set 16 and with appropriate registrationor index markers on the patient and the LINAC apparatus, then all of thephysical objects such as the patient's body, the treatment couch, andthe LINAC collimator 5 can be detected and can be mapped into thecoordinate system defined by the isocenter and the laser axes. The useof orthogonal lasers to define isocenter is commonly used in modern dayLINAC treatment setups.

[0102]FIG. 11 illustrates another embodiment in accordance with thepresent invention wherein the use of camera tracking of the patient andapparatus is associated with an image scanning apparatus as describedpreviously. As in the above description in connection with previousfigures, the patient P is on a couch top 11. The couch top 11 may haveX, Y, or Z movement, or, in the case of some CT scanners only, movementin the vertical and longitudinal directions, Y and Z. The couch top 11has optical index markers, patterns, geometric objects, or otheridentifiable structures indicated by 30, 31, and 32. The associatedapparatus 191 is shown as a toroidal scanner as for example for a CT,MRI, or PET scanner. This could be a C-shaped MRI magnet or otherconfiguration of image scan device. Typically, X-ray fields orelectromagnetic fields emanating from apparatus 191 for CT or MRIscanning are used to perform volumetric or tomographic scanning on thepatient. These fields are schematically illustrated by the dashed linesuch as 192. In accordance with the previous description, opticalindex.markers or fiducial points, illustrated for example by objects 20,21, and 23, are placed on or in proximity to the patient's skin. Asdescribed above, these could be natural landmarks, or they could beother geometric objects such as spheres, discs, pattern plates, etc.They are visible when the patient is in certain positions to the fieldof view of camera 16. In FIG. 11, only a two camera system C5 is shownwhich includes cameras 17 and 18. There is an annular, light-emittingring 17A and 17B around the cameras in the case that reflective opticalindex markers are used on the patient or the apparatus. On the CT, MR,PET, or apparatus 191 are index markers 40A, 40B, and 40C, and there maybe more according to the need. These are “visible” also to camera systemC5. Thereby the location of the imaging apparatus relative to thepatient can be determined in the 3D stereoscopic coordinate space ofcamera system C5. The video or camera processor 177 is integrated withcomparator system and couch controller 178 and/or coupled to a treatmentplanning system 36 in accord with the description above. From priorimage scan data, a target 44 may have been identified in the patient'sbody. It may be desired, according to the clinical need, that rescanningfor example in the operating room or in the treatment room is needed toassess the tissue volume near the historically determined target 44. Theimage scan machine may have a reference point indicated in FIG. 11 bythe point 187. This could be, for example, the nominal convergence pointof X-rays in a CT scanner or some calibrated geometric point in thespace of an MRI scanner reconstruction volume. Alternatively, it couldsimply be an arbitrary point which is determined by a calibrationprocess within the coordinate space of or on the image scanner. Arelationship of this reference point 187 to the external apparatus 191and its associated optical index points 44A, 44B, and 44C can beprecalibrated or determined, and therefore the camera system 16 may havein its memory storage, or in direct view, a determination of where thereference point 187 is relative to the other objects such as thepatient's body and its associated index marks 20, 21, and 23.

[0103] As one illustrated example, a patient may have been scanned by CTor MR to determine the position of a tumor in his body or his cranium.Based on that information and a treatment planning processor such as 36,surgery or other intervention may be planned. It may be desired todetermine the degree, for example, of the tumor as the resection istaking place. In this situation, a CT, MR, PET, or other scanner may beplaced in or near the operating room, and during the surgery a scan ofthe patient is required in or around the region where the tumor wasidentified by the previous imaging, and/or around the region where thesurgeon is resecting. In that case, use of the optical tracking systemas in FIG. 11 in conjunction with knowledge of a reference point(s) 192in such an interoperative scanner would enable the clinician to move thepredetermined target region 44 or interoperatively determined targetposition 44 to a region near the reference point 187 so that theinteroperative CT, MR, etc. scans will give meaningful information forits update of surgery. The use of controller system 178 coupled to couchtop 11 and the coupling to other controls of the image scanner viz.couch movement/readout would follow along the discussion above inconnection with the previous figures.

[0104] Also shown in FIG. 11 is head ring 194 attached to a patient'shead. The head ring is similar to, for example, a CRW stereotactic headring made by Radionics, Inc., Burlington, Mass., or a Mayfield headrestmade by Ohio Medical, Cincinnati, Ohio. This head ring may have indexmarkers 195, 196, and 197 on it so that its position can be tracked bythe camera system 16, and therefore the position of the head known withrespect to the reference point 187. Furthermore, by detecting theseindex markers on the head ring and also knowing the movement position ofthe couch top 11 from couch index markers such as 30, 31, and 32, thepatient's cranial anatomy can be brought into the region of the scannerin a quantifiable way by appropriate movements of couch top 11.

[0105] As is apparent to those skilled in the art, the system andprocess described above may take many forms, with a multitude ofvariations by those skilled in the art and in accordance with thepresent invention. For example, many variations of the camera form,numbers, positioning, and relative calibration are possible. Varioustypes of treatment machines such as LINACs, proton accelerators,ultrasonic machines, interventive radiofrequency devices, interventivestereotactic apparatus of all types, as well as diagnostic machines suchas CT, MR, PET, ultrasound, MEG scanners can substitute as the apparatusin the above embodiments. A variety of index markers, either surfacemounted, implanted, of geometric area type, skin bands, linear andgeometric structures taped to the skin, and so on can be used asreferencing during historic imaging and treatment or diagnosticpositioning. Various process steps can be used to implement the patienttarget positioning and movement of the patient to bring an anatomicalregion into desired relationship or relative to a predetermined positionor volume within the treatment or diagnostic machine.

[0106] In view of these considerations, and as will be appreciated bypersons skilled in the art, implementations and systems could beconsidered broadly and with reference to the claims as set forth below.

What is claimed is:
 1. A system for location of a patient's body withspatial points on a treatment or diagnostic apparatus in registrationwith image data from an image scanner, the image scanner having ascanner coordinate frame and providing the image data of at least aportion of said patient's body scanned by said image scanner to saidcomputer system to develop scanner marker coordinates in the scannercoordinate frame of scanner index markers located on said at least aportion of said patient's body, and to develop scanner targetcoordinates in said scanner coordinate frame of at least one target insaid at least a portion of said patient's body, said system comprising:a computer system to process camera data and the image data from theimage scanner; a camera system comprising two or more cameras, eachhaving a field of view that comprises at least a portion of thepatient's body on the treatment or diagnostic apparatus, said camerasystem indexing positions of the spatial points within the field ofview, having at least one reference point in a known position withrespect to said treatment or diagnostic apparatus with referencecoordinates that are known in said camera system, the camera systemproviding camera data to the computer system to develop optical markercoordinates in the camera coordinate frame of optical index markersdetectable by said camera system in the field-of-view and located in thesame position on said patient's body as said scanner index markers, andwhereby said positions of said optical index markers are known withrespect to said at least one reference point; transformation meansassociated with said computer system to transform said scanner markercoordinates to said optical marker coordinates, and whereby said scannertarget coordinates are transformed to camera target coordinates so thatthe position of said at least one target position is determined withrespect to said at least one reference point of said treatment ordiagnostic apparatus.
 2. The system of claim 1 wherein said imagescanner is a CT scanner and said scanner index markers are radiopaquemarkers that are adapted to be attached to said at least a portion ofsaid patient's body and that have positions that are detectable in saidimage data.
 3. The system of claim 1 wherein said optical index markersare light-emitting objects that are adapted to be attached to said atleast a portion of said patient's body, and emit light detectable bysaid camera system to produce detectable camera data representative ofsaid camera marker coordinates.
 4. The system of claim 1 wherein saidoptical index markers are objects with geometric patterns that aredetectable by said camera system to provide camera marker coordinates.5. The system of claim 1 wherein said optical index markers are lightreflecting objects that are adapted to be attached to said at least aportion of said patient's body and reflect light from light sourceslocated near said camera system to produce detectable camera datarepresentative of said camera marker coordinates.
 6. The system of claim1 wherein said treatment or diagnostic apparatus is a LINAC and saidreference point is a radiation isocenter of radiation beams from saidLINAC.
 7. The system of claim 1 wherein said treatment or diagnosticapparatus is a diagnostic image scanning apparatus and wherein saidreference point is a determinable point within the image acquisitionrange of the diagnostic image scanning apparatus.